Papers with verification framework
Style-News: Incorporating Stylized News Generation and Adversarial Verification for Neural Fake News Detection (2024.eacl-long)
Copied to clipboard
| Challenge: | Using generative models, the issues of producing hallucinatory contents have been raised in various domains, e.g., law, writing. |
| Approach: | They propose a style-aware neural news generator that mimics the style of real news to deceive people by identifying which publisher the style corresponds to and training a model to detect fake news. |
| Outcome: | The proposed framework outperforms state-of-the-art models in terms of fluency, content preservation, and style adherence. |
CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate grounded responses are prone to errors due to the irrelevancy of input documents. |
| Approach: | They propose a framework that leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. |
| Outcome: | Experiments on three open-domain question-answering datasets show that the proposed framework improves performance by 1.5% to 7% without any model fine-tuning. |
Where am I? Large Language Models Wandering between Semantics and Structures in Long Contexts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations of the open-domain question answering task focus solely on whether the model provides the correct answer. |
| Approach: | They propose to examine the phenomenon of discrepancies in abilities across two distinct tasks—QA and evidence selection—when performed simultaneously. |
| Outcome: | The proposed framework and resources examines the ability of large language models to perform two distinct tasks simultaneously, from the perspective of task alignment. |
Semantic Inversion, Identical Replies: Revisiting Negation Blindness in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Negation is a common occurrence in the real world and is essential for logical reasoning as it helps understand the opposite or absence of a statement. |
| Approach: | They propose a verification framework that includes task design and measurement methods to verify this phenomenon negation blindness on the query. |
| Outcome: | The proposed framework can be used to verify the model fails to capture semantic contradictions in negated queries despite its accurate understanding of knowledge about positive queries. |